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1 Introduction and Main Results 

In this paper, we consider the following second order Hamiltonian systems 

>: 

t£ ; u(t) + VV(u(t)) = (1) 



o 



with 



1 



00" -\u(t)\ 2 + V(u(t))=H. (2) 

o; 

where u G C 2 ( y R 1 ,R N ), V G C 1 ^, R 1 )- Subsequently, W(x) denotes the gradient with 
respect to the x variable, (•, •) : R x R N — > R denotes the standard Euclidean inner 
product in R and | • | is the induced norm. 

The restricted three-body problem is a reduced model for the Newtonian three-body 
problems. The existence of periodic orbits, hyperbolic orbits for this model has been 
studied by many mathematicians [1, 2, 4, 7, 8, 9, 15, 21, 23, 27] and the references therein. 
In this paper, an orbit of this problem is said to be hyperbolic if two of the three bodies 
remain bounded while the third goes to infinity with vanishing velocity. A special type 
of restricted three-body problem was considered by Sitninkov [20] and Moser [13]: Under 
Newton's law of attraction, two mass points of equal mass mi = mi = | moving in the 
plane of their elliptic orbits such that the center of masses is at rest, the third mass point 
m which does not influence the motion of the first two moving on the line perpendicular 
to the plane containing the first two mass points and going through the center of mass. 
Let u be the coordinate describing the motion of m and the center of mass of the first two 
mass points is at the origin. The restricted three-body problem consists in determining u 
such that: 

-u(t) - 



{\u{t)\ 2 + |r(t)| 2 ) 3 / 2: 
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where r(t) = r(t + 2tc) is the distance from the center of mass to one of the first two mass 
points. For a small e > 0, the function r has the form (see Moser [13]): 

r(t) = -(1 -ecost) +0(e 2 ). 

Souissi [21] used variational minimax methods and approximations to prove the ex- 
istence of at least one parabolic orbit of the circular restricted three-body problem with 
< a < 1 for 

.. , . au(t) , . 

u(t) + ^ — = 0. 3 

(Kt)| 2 +|r(t)| 2 )^ 

With < a < 2, Zhang [27] has proved 

Theorem l.l(See[27]). For (3) with < a < 2, there exists one odd parabolic or 
hyperbolic orbit which minimizes the corresponding variational functional. 

The above results are obtained for Newtonian weak force type potentials. For the two- 
body problems with charges, Wu and Zhang [26] have proved the existence of hyperbolic 
orbits for a class of singular Hamiltonian systems with fixed energy, they obtained the 
following theorem. 

Theorem 1.2(See[26]) Suppose that V G C\R N \ {O},^ 1 ) satisfies 

(Ax) V(-x) = V(x), VxeR N \ {0}, 

(A 2 ) there is a constant a G (0, 2) such that 

(x, VV(x)) = -aV{x) < for any x e R N \ {0}. 

Then for any H > 0, there is at least one hyperbolic orbit for systems (l)-(2). 

Similarly, in restricted three-body problems, we can also consider three bodies which 
are charged. Suppose e±, e 2 and e represent the charges of m 1; m 2 and m with e± — e 2 , then 
the effect force between the mass points not only obey the Newton's but also Coulomb's 
laws. When \ei\ = \e 2 \ are small enough, the first two bodies attract each other which 
implies that they can move in their elliptic orbits. The motion equation of the third mass 
point is 

mu{t) + Q(m + 2eeiM j 2 = 0. (4) 
(\u(t)\i + \r(t)\^ 

In this model, the potential is not singular which is much different from the Newto- 
nian type potentials. As to the existence of periodic orbits for non-singular Hamiltonian 
systems with fixed energy, there have been many works. In 1978, Rabinowitz [19] used 
variational methods for strongly indefinite functionals to study the existence of a periodic 
solution of a class of Hamiltonian systems on any given regular energy hyperface. He 
obtained the following result. 

Theorem 1.3(See[19]) Let H G C 1 (R 2N , R 1 ) . Suppose 

(Bi) for some b ^ 0, H^ih) is radially homeomorphic to S 2N_1 . 

(B 2 ) H z ^0, \/(EH-\b). 
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Then the Hamiltonian system 



( dH 8H \ t _ ( 
V dp ' <9g J ' J \ I 



possesses a periodic solution on H 1 (b), where z = (p,q) G R 2N , H = H{p,q), H z = 
-I\ 


/ 2Nx2N 

Since the pioneering work of Rabinowitz, there are many works on the existence of 
periodic solutions for (5) or second order Hamiltonian systems. As to the unbounded 
orbits for non-singular Hamiltonian systems with a fixed energy, there are only few paper 
relating to this topic. In 1994, E. Serra [22] has obtained the existence of Homoclinic 
orbits at infinity for a class of second order conservative systems. In his paper, he treated 
the systems with zero energy and he approximated the homoclinic orbits with a sequence 
of brake orbits which are obtained by variational methods. He obtained the following 
theorem. 

Theorem 1.4(See[22]) Suppose that the potential V G C 2 (R N ,R l ) satisfies 

(d) V{x) < for all x G R N , 

(C2) there exist Rq > 0, 7 > 2 such that 

V(x) = -j^ + W(x), W\x\>R , 

(C 3 ) ]im\ x ^ +oo W(x)\x\i = 0, 
(C 4 ) (x,VW(x)) > 0, y\x\ > R . 

Then there exists at least one solution to the problem 

' u(t) + W(u(t)) = 0, for all t G R, 

< lim t _> ±00 \u(t)\ = +00, (6) 
_ lim t _> ±00 ii(t) = 0. 

Definition 1.5 (See [4]) An orbit u(t) is called a parabolic orbit, if we have 

\u(t)\ — > +00, \u(t)\ — > as \t\ — > +00; 
An orbit u(t) is called a hyperbolic orbit, if we have 

\u(t)\ — >■ +00, |w(t)| > as |£| — > +00. 
Motivated by above papers, we have following theorems. 
Theorem 1.6 Suppose V G C 1 (i? Ar , R 1 ) satisfies 
(Vi) V(0) > = > 0, /or a// x G it^. 

(F 2 ) (x,VF(x))^0, as \x\ ->• +00. 
(V3) ^(z) -> 0, as |x| ->■ +00. 
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Then system (l)-(2) possesses at least one hyperbolic orbit for any given H > V(0). 
Theorem 1.7 Suppose V G C l (R N ,R l ) satisfies (V 2 ), (V 3 ) and 
(V 4 ) V(-x) = V(x) < 0, for all x G R N . 

Then systems (l)-(2) possesses at least one hyperbolic orbits for any H > 0. 

Remark 1 In this paper, we use the 1/2-antisymmetrical constraint to reduce the 
norm. Since the potential in this paper has no singulary, we can also reduce the norm on 
the odd-antisymmetry constrain space which is 

Er = [q G H l \ q(-t) = -q(t), \q | = \q Q | = /?} , 

where H l = W 1,2 ([— |, |], i?^). And all the proofs are similar to this paper. 

Remark 2 Notice that in model (4), if e has different sign with e±, e 2 and |e| is large 
enough, the parameter a(m + 2eei) is negative, which satisfies all conditions in Theorem 
1.6. On the other hand, if e has the same sign with e± and e 2 , the parameter a(m + 2ee±) 
is positive, which satisfies all conditions in Theorem 1.7. 

2 Variational Settings 

Let L°°([0, 1], -R^) be a space of measurable functions from [0, 1] into R N and essentially 
bounded under the following norm 

IMU°°([o,i],ii") : = esssup{\q(t)\ : t G [0,1]}. 

Similar to A. Ambrosetti and V. Coti. Zelati in [1], we use the 1/2-antisymmetrical 
constraint to reduce the norm. The space where we define the functional is as follow. 

Mr = {q G H 1 \ q(t + 1) = -q(t), \q(0)\ = \q(l)\ = R}. 

For any q G H 1 , we know that the following norms are equivalent to each other 

v V2 
l2 dt) + 



q(t)dt 

\l/2 / ,1 \l/2 



\q\\m = (£m\ 2 <Ky'+(£\Q(t)\ 2 dt 



\\q\\m = (£\m 2 dt) 1/2 + \Q(P)\- 

If q G Mr, we have / q(t)dt = 0, then by Poincare-Wirtinger's inequality, we obtain 
J o 

that the above norms are equivalent to 

/ ,1 \ 1/2 

U\m = (jf |g(t)| 2 ^) • 
Moreover, let / : M R -> i? 1 be the functional defined by 

f(q) = l -f 1 \q{t)\ 2 dtf\H-V{q{t)))dt 
I Jo Jo 

= \\\qf J\H-V{q{t)))dt. (7) 
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Then one can easily check that / G C l (M R) R 1 ) and 

</'(<?), <?(*)> = \\q\\ 2 [(H-V{q(t))-\{VV{q{t)), q {t))yt. (8) 

Firstly, we prove Theorem 1.6. To prove this theorem, our way is to approach the 
hyperbolic orbits with a sequence of periodic orbits which are obtained by the minimizing 
theory. We need the following lemma which is proved by A. Ambrosetti and V. Coti. 
Zelati in [1]. 

Lemma 2.1(See[l]) Let f(q) = \ C \q{t)\ 2 dt [\h - V(q(t)))dt and q G H 1 be 

2 Jo Jo 

such that f'(q) = 0, f(q) > 0. Set 

\f\h{t)\ 2 dt 

rp2 _ I JQ 

[\H-V(q(t))dt 
Jo 

Then u(t) = q(t/T) is a non- constant T -periodic solution for (1) and (2). 

Remark 3 In view of the proof of Lemma 2.3 in [1], we can see that the condition 

f(q) > in Lemma 3.1 can be replaced by / \q(t)\ 2 dt > 0. 

Jo 

Lemma 2.2(Palais[24]) Let a be an orthogonal representation of a finite or compact 
group G in the real Hilbert space H such that for any a G G, 

f(o--x) = f(x), 

where f G C 1 (i/, R 1 ). Let S = {x G H\ax = x,Wa G G}, then the critical point of f in S 
is also a critical point of f in H. 

Lemma 2.3(Translation Property [16]) Suppose that, in domain D C R N , we 
have a solution <fi(t) for the following differential equation 

x {n) +F{x {n ~ 1 \--- ) x) = 0, 

where x^ = d k x/dt k , k — 0, 1, • • • , n, x^ = x. Then <f>{t — t ) with t being a constant is 
also a solution. 

3 The Proof of Theorem 1.6 

Firstly, we prove the existence of the approximate solutions, then we study the 
limit procedure. In order to obtain the critical points of the functional and make some 
estimations, we need the following lemma. 

Lemma 3.1 Suppose the conditions of Theorem 1.6 hold, then for any R> 0, there 
exists at least one periodic solution on Mr for the following systems 

q(t) + VV(q(t)) = 0, VtG(-|,|) (9) 

with 

1 -\q(t)\ 2 + V(q(t))=H, V*G (10) 
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where T R is defined as 



\f\q R {t)\ 2 dt 

T^=^AJo ? (11) 

/ (H-V(q R (t)))dt 
Jo 

where q R (t) is the minimizer for the functional. 

Proof. We notice that H 1 is a reflexive Banach space and M R is a weakly closed 
subset of H 1 . By the definition of /, (Vi) and H > V(0), we obtain that / is a functional 
bounded from below and 



/(?) = \\\qf J\H -V{q{t)))dt 



(H-V(0)).. Il2 
> \\q\\ z -> +00 as ->• +00. 

Furthermore, it is easy to check that / is weakly lower semi-continuous. Then, we can 
see that for every R > there exists a minimizer q R e M R such that 

f(q R ) = 0, f(q R ) = inf /(g) > 0. (12) 

It is easy to see that ||<7k|| 2 = fo \4ii(t)\ 2 dt > 0, otherwise we deduce that \q R (t)\ = 
R > 0, on the other hand, by the 1/2- antisymmetry of q R , we have q R = 0, which is 

a contradiction. Then by Lemmas 2.1-2.3, u R [t) = q R ( T * ) : ( — — > M R is a 
non-constant T^-periodic solution satisfying (9) and (10). The proof of this lemma is 
finished. 

Remark 4 In our model, the set M R is a closed set in set H 1 . We minimize the 
functional on the set M R , however, we can not show that u R (t) solve the equation at 
±-?r. But it is true that we do not need that u R (t) is a solution at these two moments. 
Furthermore, we know that u R (t) still has definition at and \u R (±^) \ = R. 

Subsequently, we need to let R — > +00. But before doing this, we need to prove u R 
can not diverge to infinity uniformly as R — > +00, which is the following lemma. 

Lemma 3.2 Suppose that u R (t) : — ^, — > M R is the solution obtained in Lemma 
3.1, then mm. r Tr t i \u R (t)\ is bounded from above. More precisely, there is a constant 

M > independent of R such that 



min \u R (t)\ < M for all R > 0. 

fr T R T R 

tc 9. ' 9. 



Proof. Since q R G M R , it is easy to see that u R (t) = q R { T 2 ) satisfies u R (— = 



ur(^t) and it R ( — £■) = ii R (-^), then we have that 



, 4)'" 8 (y))"W"y)'"' i ("y 



-f d 
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1 R 

l^\u R (t)\ 2 + (u R (t),u R (t)))dt 
r— 

j_ 2 m 2(H - V(u R (t))) - (yV(u R (t)), u R (t))dt. 



Then we obtain that 

.la 



/ I 2H-(2V(u R (t)) + (VV(u R (t)),u R (t)))dt = 0. 

There are two cases needed to be discussed. 

Case 1. 2H - (2V(u R (t)) + (W(u R (t)),u R (t))) = 0, which implies that 



2H = 2V(u R (t)) + (VV(u R (t)),u R (t)), a.e. te 



' 2 ' 2 



Hypotheses (V2), (V3) imply that there exists a constant Mi > independent of R such 
that 



te 



min \u R (t)\ < Mi. 



2 ' 2 



Case 2. 2(if-V(ufl(t)))-(VV(u fl (t)),iifl(t)) changes sign in 
exists t E ~^f-,^f suc h that 

2H-(2V(u R (t )) + (W(ufl(to)W(to))) < 0, 



2 ' 2 



. Then there 



which implies that 



2tf < 2V(u R (to)) + (VV(u R (t )),u R (t )). 



It follows from H > and hypotheses (V2), (V3) that there exists a constant M 2 > 
independent of R such that 



te 



min |«r(£)| < M 2 . 



Then the proof is completed. 



Lemma 3.3 Suppose that R > M and u R (t) is the solution for (9) — (10) obtained 
in Lemma 3.1, where M is from Lemma 3.2. Set 



and 



t + = sup < t G 



*_ = inf it e 



Tr Tr 
' 2 ' 2 

Tr Tr 
' 2 ' 2 . 



\u R (t)\ < L 



\u R {t)\ < L 



where L is a constant independent of R such that M < L < R. Then we have that 
T R T R 

— — t+ — > +00, t- + — > +00 as it! — > +00. 
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Proof. By the definition of u R (t) we have that 













Ur (t) 









R. 



Then, by (V\) and the definitions of t + , we have 

£*~ y/H - V(u R {t))\u R (t)\dt > J~*~ y/H-V(0)\u R {t)\dt 



> - V(0) I 2 \u R (t)\dt 

u R (t)dt 



> JH-V(0) 



> JH -V(0)(R- L). 



(13) 



It follows from Lemma 3.1 and (Vi) that 

f .lR 

J 2 y/H-V(u R (t))\u R (t)\dt = V2 



H-V(u R (t))JH-V(u R (t))dt 



Combining (13) with the above estimation, we obtain that 

'T R 



yjH-V(0){R-L) < V2H(^-t 



Then we have 



T 



— t + — > +oo, as R — > +oo. 



The limit for t- + can be obtained in the similar way. The proof is completed. 
The Limit Procedure Subsequently, we set that 

Til Tr 
' 2 ' 2 . 

and 



t* = inf u G 



\u R {t)\=M 



u R (t)=u R (t-t*) 
Since L > M, we can deduce that t+ > t* > t_, which implies that 

T R 



T 

R i _#* 

— 2~ +t ->■ -oo, 



+ t* — > +oo as R — > oo. 



Then it follows from (13) that 

l\u* R (t)\ 2 +v( UR (t))=H, vtG ^+ t *, 7 f+t*y 

which implies that 

\u R (t)\ 2 = m-v(u R (t))), vtG (-Y +r 'T +r )- 



8 



By (V3) and V £ C X (R N , R 1 ), we can deduce that there exists a constant M 4 > inde- 
pendent of R such that 

\V(u* R (t))\<M 4 for all t e (-^ + f, + f) . 
Then there is a constant M 5 independent of i? such that 

\u* R (t)\<M 5 forall (~y + **' T + **) " 



which implies that 

K(*i) -«*(*») | < 



f 1 ii R {s)ds < f 1 \u* R (s)\ ds < M 5 \t ± - t 2 \ 
Jt 2 Jt 2 



for each i? > and t 1? t 2 £ ( — + £*,^jf + which shows is equicontinuous. 

Then there is a subsequence {u* R } R>0 converging to in C^R 1 , R N ). Then there 
exists a function u^t) such that 

(i) u* R (t) ^ Uoo (t) in C loc {R\R N ) 

(ii) |-u 00 (t)| —7- +00 as |t| — >■ +00 

and Uocit) satisfies systems (1) — (2). 

From the above lemmas, we have proved there is at least one hyperbolic solution for 
(1) — (2) with H > 0. We finish the proof of Theorem 1.6. 



4 The Proof of Theorem 1.7 

Since the potential in Theorem 1.7 is negative and of C 1 class, the proof of this theorem 
is more simple. Similar to the proof of Theorem 1.6, we consider the functional (7) on 
M R which is / : M R ->■ R. 

Lemma 4.1 Suppose the conditions of Theorem 1.5 hold, then for any R > 0, there 
exists at least one periodic solution on M R for the following systems 

?(*) + W(?(*)) = 0, Vfe(-^,^) (14) 

with 

1 -\q(t)\ 2 + V(q(t)) = H, Vie (~>lf)- (15) 



Proof. We notice that H 1 is a reflexive 
subset of H 1 . Since H > 0, we obtain that 

/(?) = \m 2 £(h- 

which implies that / is a functional bounded i 
that / is weakly lower semi-continuous and 

f(q) — > +00 as 



Banach space and M R is a weakly closed 
V(q(t))dt>^\\q\\ 2 , 

rom below, furthermore, it is easy to check 
q\\ — > +00. 
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Then, we conclude that for every R > there exists a minimizer Q R e Mr such that 
/'(<?*) = 0, f(Q R ) = inf /(g) > 0. 

It is easy to see that ||<5i?|| 2 = Jq \Qn(t)\ 2 dt > 0, otherwise we deduce that | | = 
R, on the other hand, by the 1/2-antisymmetry of Qr, we have Qr = 0, which is a 
contradiction. This implies that /(Qr) > 0. Then let 



dt 



(H-V(Q R (t)))dt 



(16) 



then by Lemmas 2.1-2.3, U R (t) = Q R ( t+ T * ) : (— -rf, ^f) — >■ M R is a non-constant T R - 
periodic solution satisfying (14) and (15). The proof of this lemma is finished. 

Subsequently, we need to show that U R (t) can not diverge to infinity uniformly as 
R — > +oo. Moreover, we prove the following lemma. 



Lemma 4.2 Suppose that U R (t) : 



2 ' 2 



— > M R is the solution obtained in Lemma 



4-1, then min 



Ir Ir 

2 ' 2 



\U R (t)\ is bounded from above. More precisely, there is a constant 



M' > independent of R such that 

min \U R (t)\ < M' for all R>0. 



±R ±R 

2 ' 2 



The proof of this lemma is same with that of Lemma 3.2. 

Lemma 4.3 Suppose that R > M' , where M' is defined in Lemma 4-2 and UR(t) is 
the solution for (14) and (15) obtained in Lemma 4-1- Set 



t + = sup < t E 



Tr Tr 
' 2 ' 2 



\U R (t)\<l} 



and 



t- = inf it e 



Tr Tr 
' 2 ' 2 . 

where I is a constant independent of R such that M' < I < R. Then we have that 



\U R (t)\<l} 



Tr Tr 

t+ — > +00, t- H > +00 as R — >■ +00. 



Proof. By the definition of U R (t) we have that 



U R {- 



R 



u 



R 



2 



R. 



Then, by (V4) and the definitions of t + and t_, we have 

(~ JH -V{U R {t))\U R {t)\dt > Vh[^\i 
Jt+ Jt+ 



> 



U R (t)dt 



> VH(R-l) 



(17) 
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and 



H-V(U R (t))\U R (t)\dt > VH 



\U R (t)\dt 



> 



Ik 

2 



U R (t)dt 



> VH(R-l). 



(18) 



Since V £ C l (R N , R 1 ), it follows from (V3), that there exists a constant M 6 > indepen- 
dent of R such that 



\V(U R (t))\ < M e for all t E 



Tr Tr 

' 2 ' 2 



which implies that 



Ir 

2 



H-V(U R (t))\U R (t)\dt = y/2 I ' (H-V(U R (t)))dt<V2(H + M 6 )(-£-t + ). 



T 



Combining (17) with the above estimates, we obtain that 



y/H(R -L)< V2(H + M 6 ) (-j - t, 



Then we have 



— t+ — > +00, as R — > +00. 



The limit for t- + ^ can be obtained in the similar way. The proof is completed. 
The following limit procedure is similar to the proof in Theorem 1.6. □ 
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